Qresp 2.0 modernization, auth, ownership, drafts, and admin management - #68

…ent/develop

New Release, v2.0.5

DEPENDENCY_AUDIT.md records latest-stable versions (PyPI/npm, 2026-07-02),
risk, and breaking changes for every dependency before any edits.

requirements.txt / setup.py: remove every declared-but-never-imported package
(Flask-API, Flask-HTTPAuth, flask-profiler, Flask-WTF, paramiko, schedule,
py3dns, pyasn1, validate-email, pyOpenSSL, swagger-spec-validator, coveralls,
python-dateutil, expiringdict, cffi, cryptography) and redundant explicit
transitives (itsdangerous, Jinja2, urllib3). Add `requests` (used by
project/util.py, was transitive-only). setup.py: python_requires >=3.10
(jsonschema/coverage/gunicorn already require it), test tools moved to a
`test` extra. Flask<2.3 / connexion<3 caps unchanged in this phase.

Verified (fresh CPython 3.11.5 venv): pip check OK, GET / -> 200, nose2 17 OK.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…e B)

flask-mongoengine 1.0 is unmaintained and hard-blocks Flask>=2.3 (it uses the
removed flask.json.JSONEncoder APIs). The models are plain mongoengine
Documents; the extension was only a connection shim, so:
- project/__init__.py: mongoengine.connect() straight from Config settings;
  username/password only passed when configured. Drops the write-only
  app.config['MONGODB_*'] mirror (never read anywhere).
- project/db.py: MongoDBConnection re-points via mongoengine.disconnect() +
  connect() (default alias cannot be silently reused with new settings);
  catches both pymongo and mongoengine ConnectionFailure.
- requirements.txt / setup.py: drop flask-mongoengine (also drops its
  Flask-WTF / WTForms[email] / email-validator transitive chain; views.py uses
  EmailField for rendering only, no Email() validator).

Verified (fresh CPython 3.11.5 venv): pip check OK, flask-mongoengine absent
from the tree, GET / -> 200, nose2 17 OK. Real-MongoDB round-trip re-verified
in the Docker phase.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Connexion 3 keeps FlaskApp but runs routing/validation/swagger-ui as ASGI
middleware around Flask, so the servable object changes:
- __init__.py: FlaskApp(jsonifier=Jsonifier(cls=MongoJSONEncoder)); swagger.yml
  resolved relative to the package (cwd-independent); app.json provider set.
- NEW project/jsonutil.py: restores mongoengine-document JSON serialization
  that flask-mongoengine's patched encoder used to provide, in the identical
  bson json_util shape, for BOTH layers (Connexion jsonifier for /api/*,
  Flask JSON provider for routes.py jsonify()).
- api.py: `from connexion import request, jsonifier` (gone in v3) -> flask
  request proxy; jsonifier import was unused.
- run.py / __main__.py: serve via connexionapp.run() (uvicorn) so dev serving
  includes the middleware; __main__ gains the main() the qresp console script
  always pointed at (previously broken entry point).
- compose: gunicorn now uses -k uvicorn_worker.UvicornWorker on
  project:connexionapp; dev compose serves uvicorn --reload (flask run would
  bypass middleware). deps: connexion[flask,swagger-ui,uvicorn]>=3.3 +
  uvicorn-worker; caps dropped.
- NEW tests/test_api_endpoints.py: 9 tests through the real ASGI middleware
  (validation 400, Swagger-2 body-name mapping into `req`, EmbeddedDocument
  serialization on /api/paper/{id}, Flask passthrough, /api/ui/).

Flask stays 2.2.5 in this phase (cap lifted next, Phase C).
Verified (fresh CPython 3.11.5 venv): pip check OK, GET / -> 200,
nose2 26 tests OK (was 17).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

With Connexion 3 in and flask-mongoengine gone, the last <2.3 caps are
removed: Flask 2.2.5 -> 3.1.3, Werkzeug 2.2.3 -> 3.1.8. App code needed no
changes for removed Flask APIs (no before_first_request / JSONEncoder /
flask.escape usage); flask-sitemap 0.4.0, Flask-Session 0.8.0, flask-cors
6.0.5 all verified working on Flask 3 (boot + /sitemap.xml render).

Two latent bugs exposed by the new page-render tests (both pre-existing on
the current WTForms 3.2.2 baseline, not caused by Flask 3):
- views.py RequiredIf: WTForms-2 tuple field_flags crashed every form that
  binds it ("'tuple' object has no attribute 'items'" on /qrespcurator);
  now a dict, and the broken super(RequiredIf).__init__() call fixed.
- util.py Servers: the federated-servers registry was fetched with no
  timeout and no error handling, so an outage or non-JSON reply 500'd
  /qrespcurator + /qrespexplorer (reproduced live against
  paperstack.uchicago.edu today); now degrades to an empty list.

tests: +3 page-render tests (curator page mocks the registry fetch to stay
hermetic). Verified (fresh CPython 3.11.5 venv): pip check OK, GET / -> 200,
nose2 29 tests OK.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…se F)

- requirements.lock.txt: regenerated from the verified clean venv. 81 -> 65
  pins: Flask 3.1.3, Werkzeug 3.1.8, connexion 3.3.0 (+starlette/httpx/
  a2wsgi/uvicorn/uvicorn-worker), flask-mongoengine chain and all unused
  packages gone. Header documents Windows provenance (uvloop absent -> Docker
  uvicorn falls back to asyncio; harmless).
- backend/Dockerfile: python:3.11-slim -> python:3.14-slim. Verified
  in-container: lock installs, gunicorn -k UvicornWorker serves /, /api/search,
  /api/ui/ (200); DAO insert->read against mongo:4.4; nose2 29 OK.
- backend/Dockerfile.dev: python:3.6-alpine (EOL + apk toolchain) ->
  python:3.14-slim, mirroring the prod image. Verified: build + uvicorn
  --reload serves / and /api/search (200) against the dev db.
- docker-compose.dev.yml: dev db mongo:3.6.18-xenial -> mongo:4.4. 3.6 is EOL
  and PyMongo 4.17 requires MongoDB >= 4.0, so the old dev db could no longer
  connect at all. Production mongodb default stays mongo:4.4 (unchanged).
- CI backend-smoke: python matrix 3.11 (dev baseline) + 3.14 (Docker runtime);
  all YAML validated locally.
- tests: assertEquals -> assertEqual (unittest aliases removed in Python 3.12;
  was 16 in-container errors on 3.14).
- .coverage untracked + gitignored (binary artifact rewritten by every run).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

FULL_STACK_MODERNIZATION_REPORT.md rewritten for wave 2: dependency table
(before/after), changed-files list, code-migration summary, full verification
matrix (local venvs, lock reproducibility, Docker prod/dev on py3.14, real-
MongoDB round trip, validation-middleware checks), latent bugs fixed,
deployment risks, and exact next steps. QUICKSTART/TROUBLESHOOTING: serving
commands updated for ASGI (uvicorn / gunicorn -k UvicornWorker), test counts
17 -> 29, stale Docker/Mongo status rows corrected.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…st 30

Toolchain: Node 24.18.0 / Yarn 1.22.22 (Yarn 1 kept -- minimal churn).
Verified locally: yarn install OK, yarn build OK (Next 16.2.10, Turbopack,
7 routes), yarn test OK (RTL, 2 suites / 5 tests).

Dependency majors (see DEPENDENCY_AUDIT.md for the audit):
- next 9.4.4 -> 16.2.10; react/react-dom 16.13.1 -> 19.2.7
- @material-ui core v5-alpha + icons/lab v4 mix -> @mui/material 9.1.2 +
  @mui/icons-material 9.1.1 (+ @mui/material-nextjs, emotion incl.
  @emotion/server); @mui/lab dropped (Alert/Autocomplete/Pagination in core)
- react-hook-form 6 -> 7.80; @hookform/resolvers 0.1 -> 5.4; yup 0.29 -> 1.7
- jest 26+enzyme -> jest 30 + next/jest + React Testing Library
- axios 0.19 -> 1.18; ajv 6 -> 8 (strict:false, draft-07 schema unchanged)
- simple-react-lightbox (dead, React 16-only) -> yet-another-react-lightbox
- vis-network 7 -> 10 (standalone import; hammerjs/keycharm/component-emitter
  peer shims dropped); fontawesome 5 -> 7; react-checkbox-tree 1.6 -> 2.0

Code migrations:
- JSS -> emotion: 12 makeStyles/withStyles files converted to styled()/sx;
  _app/_document rebuilt on @mui/material-nextjs v16-pagesRouter (replaces
  ServerStyleSheets + jss-server-side removal); createTheme.
- MUI v4 API sweep: justify->justifyContent (20), Hidden -> responsive sx
  (4 files), Accordion TransitionProps -> slotProps.transition, Popover
  PaperProps/classes -> slotProps/sx, visuallyHidden util for sort labels.
- RHF v7: register-as-ref removed -- TextInput/NameInput/RadioInput wrappers
  register internally (callers pass register=); Controller as= -> render;
  errors moved to formState.errors (13 forms); unregister(object) -> (name);
  bracket field names (PIs[0].x) -> dot syntax; touched -> touchedFields.
- yup 1: when() then/otherwise now function form (ToolsInfoForm).
- next/link: no child <a>; MUI Buttons render component={Link}; href
  object replaces as=; styled-jsx anchors rescoped via :global().
- React 19: CSSTransition needs nodeRef (findDOMNode removed) -- FadeTableRow
  wrapper keeps the row fade; Turbopack: imported-binding reassign fixed
  (explorer.js); @mui/icons-material 9 dropped *Outline aliases ->
  *Outlined equivalents.
- Enzyme specs rewritten with RTL/user-event; babel-jest/.babelrc removed
  (SWC via next/jest), custom cssTransform deleted.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Next 16 requires Node >= 20.9; Node 24 matches the verified local toolchain
(local yarn build + test green before this change, per the required order).
pm2 no longer needs the Node-14 pidusage pin.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Frontend section rewritten from "blocked on Node 14" to DONE: Next 16.2.10 /
React 19.2.7 / @mui 9 / RHF 7 / jest 30+RTL on Node 24, build+tests verified;
deployment risks now include the gui image rebuild and a staging click-through
list (forms, lightbox, workflow graph, visual parity); next steps updated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Five goals (modernization done; UI regression repair; Google identity-only
auth; owner/admin edit+deactivate; agentic literature explorer) with MVP/defer
splits, ordering, file map, risk list, and branch plan. No code changes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

MUI v9 removed two APIs the whole layout was built on, and both fail
SILENTLY (unstyled div / ignored props), which broke the staging UI vs
https://paperstack.uchicago.edu:
- Box system props (display/flexDirection/flexGrow/alignItems/m/p/
  fontWeight/...) emit zero CSS in v9 (class `css-0`) -> header/nav layout,
  home hero spacing, action-button rows, footer logo rows and all bold-text
  Box wrappers rendered as plain block divs.
- The legacy Grid API is gone: `item` leaks to the DOM and xs/sm/md sizes
  are ignored -> every grid cell (forms, tables, footer rows, curator
  layout) lost its sizing.

Fix (mechanical, no redesign, values preserved verbatim):
- 47 Box tags: system props -> sx={{ ... }} (dynamic expressions kept).
- 156 Grid tags across 27 files: item dropped, xs/sm/md/lg -> size={n} /
  size={{ xs: .., sm: .. }} (bare xs -> size="grow").

Verified: yarn build OK (7 routes), yarn test OK (5 tests), zero legacy
props left (grep), and a jsdom computed-style probe confirms Layout root
is flex-column again, footer rows are flex/space-evenly/32px padding,
home action row is centered with m=1 wrappers, and Grid emits size
classes (MuiGrid-grid-xs-12 etc). Browser-level comparison against the
working site remains for staging QA.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Backend-only development auth endpoints (Qresp 2.0 checklist goal 3,
identity skeleton — no ownership, no permissions, no Google yet):
- GET  /api/auth/me         -> {authenticated, user} from the Flask session
- POST /api/auth/logout     -> clears only session["auth_user"]
- POST /api/auth/dev-login  -> dev/staging-only login; body {email required,
  name defaults to email, is_admin defaults false}; email trimmed+lowercased

Session shape (no secrets): session["auth_user"] =
  {email, name, is_admin, provider: "dev"}.

Safety gate: dev-login is OFF by default and checked per request —
enabled only when QRESP_ENABLE_DEV_LOGIN (env, via the existing
Config.get_setting override; optional [AUTH] config.ini fallback) is
1/true/yes/on; otherwise the endpoint returns 404 JSON. Production runs
without the variable. (app.config['env'] is hardcoded 'DEV' even in prod,
so the gate deliberately does NOT trust it.)

Routes are spec-first in swagger.yml (Connexion validates the body;
missing email -> 400). New project/auth.py module keeps /me and /logout
provider-agnostic for the upcoming Google OAuth swap-in.

tests: +6 in tests/test_auth.py through the real ASGI middleware with
cookie round-trip (anonymous /me; login->me->logout->me; name default;
admin flag; email validation 400s; disabled-by-default 404). No MongoDB
needed. Frontend untouched.

Verified: boot GET / -> 200; nose2 35 tests OK (was 29).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Ownership MVP on top of the session skeleton (no update API yet, no
frontend, no Google):

- models.py: Paper.owner_email (StringField) — the VERIFIED session email
  of the publisher, distinct from the curator-declared
  info.insertedBy.emailId. Absent on all legacy records => "ownerless".
- auth.py helpers: get_current_user(), is_admin() (QRESP_ADMIN_EMAILS
  env/config allowlist, plus the session is_admin claim from dev-login),
  can_edit_paper(paper, user) -> (allowed, reason), stamp_owner(payload).
  Rules: anonymous no; admin yes; owner yes; ownerless -> admin only;
  everyone else no.
- api.py publish handler stamps owner_email from the session BEFORE the
  payload is validated/stored, so ownership survives the email-verification
  round trip into insertIntoPapers (schema has no additionalProperties cap;
  Paper(**data) accepts the new field). Anonymous publishing stays allowed
  and simply yields an ownerless record — no existing publish flow or test
  breaks.
- NEW GET /api/paper/{id}/permissions (spec-first in swagger.yml; singular
  /paper/{id}/... to match the existing routes) returning {can_edit,
  reason, owner_email, authenticated, is_admin} so the frontend can
  show/hide edit controls later; the same can_edit_paper rule will guard
  the future update/deactivate endpoints. Unknown id -> 404.

tests: +9 (tests/test_permissions.py, mongomock + real ASGI middleware):
anonymous/owner/non-owner/admin on owned records, ownerless admin-only,
session admin flag, 404, can_edit_paper unit matrix, stamp_owner with and
without a session.

Verified: boot GET / -> 200; nose2 44 tests OK (was 35). Frontend untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Connects the frontend to the existing backend auth/permission MVP
(commits 1678e07 + edc9279). No Google OAuth, no update API, no redesign.

- NEW Context/Auth (AuthState/authReducer/authContext + types), following
  the existing Alert/Loading context pattern. State: {loading,
  authenticated, user{email,name,is_admin,provider}, error}; helpers
  refresh()/devLogin()/logout(). GET /api/auth/me on app load. All calls
  use relative same-origin /api paths so the browser carries the Flask
  session cookie itself; nothing is stored in localStorage.
- NEW components/AuthControls.js in the header links row (desktop row and
  mobile drawer share the fragment): authenticated -> name/email
  (+"(admin)") and Sign out; anonymous -> "Dev sign in" opening a small
  dialog (email, optional name, dev-only admin checkbox) that POSTs
  /api/auth/dev-login and shows "Development login is unavailable on this
  server." when the backend gate returns 404. Explicitly labelled
  staging/development-only; no Google branding or scopes.
- NEW components/Paper/PermissionNotice.js on paperdetails (skipped for
  previews): fetches GET /api/paper/{id}/permissions (backend decision,
  never frontend logic; refetches when auth state changes) and renders a
  small notice - "You can edit this record (owner|admin)" / "Sign in to
  edit this record" / "Only the record owner or an admin can edit this
  record". Fetch failure (previews, older backend) renders nothing. No
  edit button yet - there is no update API to point it at.
- Publish/session behavior: NO change needed - publish/preview already go
  through getServer() = window.location origin, i.e. same-origin absolute
  URLs, so session cookies are attached automatically.

tests: +8 (AuthControls anonymous/authenticated/logout/disabled-gate;
PermissionNotice owner/anonymous/non-owner/silent-failure) with axios
mocked. Verified: yarn build OK (7 routes), yarn test 13 OK (was 5),
backend nose2 44 OK (untouched).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Smallest vertical slice proving owner/admin editing end to end (no Google,
no curator redesign, no delete):

Backend — PUT /api/paper/{id} (same singular path as the existing GET,
spec-first in swagger.yml):
- 404 unknown id; can_edit_paper gate -> 401 anonymous / 403 authenticated
  non-owner (ownerless records stay admin-only).
- Top-level payload fields are merged into the stored document
  (existing.to_mongo + payload), filtered to defined Paper fields, then
  re-validated through the Paper model constructor and saved under the
  same pk -> embedded docs coerce, required fields stay enforced (400 on
  violation).
- Server-owned fields can never come from the payload: id/_id,
  owner_email (forced back from the stored record), version/versions.

Frontend — PermissionNotice grows the minimal edit flow:
- "Edit metadata" button only when the BACKEND permission decision says
  can_edit (no frontend-only logic).
- MVP dialog edits one harmless field (tags, comma separated) and PUTs
  /api/paper/{id}; success reloads the page so getServerSideProps
  refetches; 401/403 show the backend reason; other failures show a
  generic error. Full curator-integrated editing is a later phase.

tests: backend +8 (test_update_paper.py on the shared permission fixture:
anonymous 401, non-owner 403, owner/admin persist, ownerless admin-only,
owner_email immutable, 404, invalid payload 400); frontend PermissionNotice
suite extended to 6 (edit action shown/hidden by backend decision, PUT
payload + reload, forbidden save shows backend reason, silent failure).

Verified: nose2 52 OK (was 44); yarn build OK; yarn test 15 OK (was 13).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Google becomes the real identity provider on top of the existing session
and permission system. Identity ONLY: scopes are hardcoded to
openid/email/profile in project/auth.py (never read from config), so this
flow can never request Drive/Gmail/other Google APIs.

Backend (spec-first, requests_oauthlib which was already a dependency):
- GET /api/auth/google: builds the consent URL, stores the OAuth state in
  the session, 302 to Google. 503 JSON when unconfigured -- the app still
  boots and dev-login keeps working.
- GET /api/auth/google/callback: rejects provider errors, missing flows
  and state mismatches (400); exchanges the code and fetches userinfo
  server-side; tokens are used for that single call and discarded (never
  stored in the session or exposed to the frontend). Stores the existing
  session shape: {email (trimmed+lowercased), name (defaults to email),
  is_admin, provider: "google", google_sub}. is_admin comes EXCLUSIVELY
  from the QRESP_ADMIN_EMAILS / [AUTH] ADMIN_EMAILS allowlist -- Google is
  never trusted for roles. 302 back to "/" -> AuthState refetches /me.
- Config via existing conventions: QRESP_GOOGLE_CLIENT_ID /
  QRESP_GOOGLE_CLIENT_SECRET / QRESP_GOOGLE_REDIRECT_URI env overrides,
  [GOOGLE_API] config.ini fallback; the section's existing
  auth_uri/token_uri/user_info endpoint entries are honored with Google
  defaults otherwise. No secrets committed.
- /auth/me, /paper/{id}/permissions and PUT /paper/{id} work unchanged on
  a Google session (same auth_user shape).

Frontend: anonymous header now shows "Sign in with Google" (plain text,
no Google branding) navigating to /api/auth/google; "Dev sign in" stays
as the staging tool. No redesign.

tests: backend +7 (mocked OAuth2Session, no network: unconfigured 503 +
dev-login unaffected; redirect carries identity-only scopes + state;
state mismatch and cold-callback 400; user stored with provider google
and normalized email; allowlist -> is_admin; provider error 400).
frontend +1 (Google link href for anonymous users).

Verified: nose2 59 OK (was 52); yarn build OK; yarn test 16 OK (was 15).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Hardening/readiness only — no new product features.

1) OAUTHLIB_INSECURE_TRANSPORT is no longer hardcoded on (it silently
   weakened production OAuth). It is now set ONLY when explicitly enabled
   via QRESP_OAUTHLIB_INSECURE_TRANSPORT (env, or [AUTH] ini) for local/
   plain-HTTP development. HTTPS staging/production need nothing; local
   tests never exchange real tokens, so they pass without it.

2) Minimal CSRF protection (double-submit header, session-bound):
   - /api/auth/me now also issues csrf_token (secrets.token_urlsafe,
     stored in the session, constant-time compared).
   - X-CSRF-Token is REQUIRED on mutating routes when the request carries
     an authenticated session: POST /api/auth/logout, POST /api/publish,
     PUT /api/paper/{id}. Anonymous API/CLI usage (incl. anonymous
     publish) is unchanged; dev-login stays unwrapped by design (it only
     establishes a session; Google login is protected by OAuth state).
   - Frontend: AuthState captures the token from /me and an axios request
     interceptor attaches it to MUTATING SAME-ORIGIN requests only (covers
     the publish/preview calls built on getServer(); never leaks the
     header to external hosts like the DOI scraper).

3) Google return path: /api/auth/google?next=<path> remembers a validated
   same-origin, path-only target (no scheme/host, no //, no backslash) in
   the session; the callback re-validates and redirects there instead of
   "/". The header button now passes the current page (router.asPath).
   Open redirects are covered by tests.

4) Staging readiness docs: NEW STAGING_QA_CHECKLIST.md (env vars, curl
   smoke, browser matrix incl. mobile drawer; qresp_staging only, no
   secrets committed); QRESP_2_IMPLEMENTATION_CHECKLIST.md gains a
   pre-production blockers section (done: transport gate, CSRF, open
   redirect; open: id_token verification, cookie flags via nginx, rate
   limiting, unset dev-login in prod, util.py verify=False).

tests: backend +4 / updated for CSRF (logout/PUT require the token; next
round-trip; unsafe-next fallback) -> nose2 63 OK (was 59). frontend:
AuthControls spec covers the next-carrying Google href with next/router
mocked -> yarn test 16 OK; yarn build OK.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

/curator crashed client-side with "TypeError: findDOMNode is not a
function" (performExit -> updateStatus -> componentDidUpdate). Root
cause: components/switchFade.js used react-transition-group's
<Transition> WITHOUT nodeRef, so it fell back to ReactDOM.findDOMNode --
removed in React 19 -- the moment SwitchTransition (mode="out-in") exited
a side. All six CuratorElements (PaperInfo/Reference/License/FileServer/
Documentation/Curator) render through SwitchFade, taking the whole
curator page down.

Fix: FadeTransition now owns a nodeRef attached to the div it renders and
passes it to <Transition> -- the same pattern as the FadeTableRow fix in
Table/Table.js from the modernization wave. Visuals/timings unchanged.
(The wave-3 audit grepped for CSSTransition|TransitionGroup and missed
this lone `Transition` import; a fresh repo-wide react-transition-group
audit confirms switchFade.js and Table.js are the only two usages, both
now nodeRef-correct.)

tests: NEW SwitchFade.spec.js -- toggling form<->display, which throws
under React 19 without the nodeRef, now passes (+2, yarn test 18 OK).
yarn build OK. Backend untouched.

Not addressed here (observed on staging, reported separately): the MUI
Dialog aria-hidden focus warning (cosmetic), and the
ERR_TLS_CERT_ALTNAME_INVALID from the qresp.hybrid3.duke.edu federated
node (its cert is for materials.hybrid3.duke.edu; search.js already
try/catches per server+endpoint, so /search degrades instead of dying).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

paperdetails showed "Error Getting Paper Data" on staging: its
getServerSideProps fetched `${query.server}/api/paper/{id}` with
query.server = https://localhost:8443 — but SSR runs INSIDE the gui
container, where localhost is the container itself, not the host tunnel
/nginx, so the fetch could never reach the backend.

Fix: NEW Utils/serverSideApi.js resolveServerSideApiBase(ctx, server) —
SSR-only decision of the fetch base:
- external federation nodes (host differs from the request host and is
  not local): used as-is, unchanged behavior;
- missing server, localhost/127.0.0.1/::1/*.localhost, or same host as
  the incoming request (honoring x-forwarded-host): rewritten to
  QRESP_INTERNAL_API_URL (e.g. http://backend:5000); without the env var
  it falls back to the original behavior;
- unparseable or non-http(s) input (ftp:, javascript:, //host, ...) is
  never used as a fetch target (internal or null -> the page's existing
  error path).
The env var is deliberately not NEXT_PUBLIC_* and the public query.server
passed to components (file-server links, chart paths) is untouched;
preview fetches go through the same resolver.

docker-compose.yml: gui gains QRESP_INTERNAL_API_URL=http://backend:5000
and joins the backend network so SSR can reach the backend service.

tests: +9 (resolver matrix: localhost/same-origin/x-forwarded-host ->
internal; external unchanged; missing -> internal or null; malicious
schemes neutralized; env-unset fallbacks) -> yarn test 27 OK (was 18);
yarn build OK; compose YAML validated. Backend untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

paperdetails SSR returned Next 500 ("TypeError: D is not iterable") even
after the API base fix: React 19 no longer applies .defaultProps on
FUNCTION components, so ChartInfo/DatasetInfo/ToolsInfo/ScriptsInfo
received editColumn = undefined from the page (which never passes it) and
crashed on `...editColumn` when building their table columns during SSR.

Fix: every function-component .defaultProps in the frontend (16 sites,
none remain) moved into destructuring default parameters — behavior and
UI unchanged:
- crash class (iterable spread on paperdetails): Charts, Datasets, Tools,
  Scripts (editColumn = [], inDrawer = true, showSlider/showWorkflows);
- same failure class nearby: Workflow/Graph (manipulate = {}),
  Workflow/Legend, pages/paperdetails ({preview = false});
- shared/benign but same regression: drawer (defaultOpen), labelvalue
  (SimpleLabelValue/LabelValue), Form InputFields/NameInput/RadioInput/
  SelectInput/TextInput (type moved out of ...rest and passed explicitly)
  and Form/Util SubmitAndReset.

tests: NEW PaperInfoDefaults.spec.js renders ChartInfo/DatasetInfo/
ToolsInfo/ScriptsInfo with ONLY their required props (exactly how
paperdetails renders them) — these throw "not iterable" before this fix
and pass after (+4, yarn test 31 OK; yet-another-react-lightbox is
ESM-only so the suite mocks it, and the router mock provides events for
LoadingState). yarn build OK. Backend/Docker/auth untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Owners/admins can now edit a whole record through the EXISTING curator
forms: paperdetails -> "Edit in Curator" -> /curator?edit=<id>&server=...
-> forms pre-populated from the stored document -> Save Changes -> PUT
/api/paper/{id} -> back to paperdetails. Create/publish mode is untouched
(TopActions/Publish render exactly as before when ?edit is absent).

Backend:
- NEW GET /api/paper/{id}/raw (spec-first): the edit flow's data source.
  Gated by the same can_edit_paper rule as updates (401 anonymous / 403
  non-owner / 404 unknown; ownerless -> admin only), returns the stored
  document with _id/owner_email stripped. The public display-shaped read
  stays GET /api/paper/{id}.
- PUT /api/paper/{id} unchanged (already merge + Paper._fields allowlist +
  model re-validation + id/_id/owner_email/version/versions blocked);
  now covered by full curator-payload tests.

Frontend:
- Utils/model.js: convertReqSchematoState (previously dead code) fixed and
  hardened -- referenceUtil.set takes an object (was positional -> would
  have produced "undefined ..." publication), journal.fullName unwrapped,
  Person objects trimmed to the name triple before namesUtil.set, legacy
  records with missing sections tolerated. NEW convertStateToUpdatePayload:
  publish payload + fields the curator does not manage preserved from the
  original document (info.downloadPath/gitPath/isPublic/..., original
  schema URL); identity/server-owned fields stripped client-side and
  enforced server-side regardless.
- NEW CuratorElements/EditMode.js: EditModeController (backend permission
  gate -> loads /raw -> setAll into existing curator state; unauthorized
  users get a clear message and NO forms/save controls) + SaveChangesBar
  (reuses Publish's validate(), PUTs with the session CSRF token via the
  existing axios interceptor, Cancel/Save, returns to paperdetails).
- pages/curator.js: EditModeController wraps the existing element tree;
  edit mode hides TopActions and swaps Publish for Save Changes.
- PermissionNotice: the MVP tag-edit dialog is REPLACED by the single
  "Edit in Curator" entry point (no two competing edit paths).
- Publish.js: validate() exported for reuse, and its ajv step no longer
  crashes -- ajv 8 THROWS compiling schema_v1.2.json (duplicate
  draft-04-style `id` anchors make "#/properties/collections/items"
  ambiguous), which had silently broken the publish button since the ajv
  upgrade; compile failures now log and fall through (backend re-validates
  every payload anyway).

tests: backend +10 (test_edit_flow.py: /raw permission matrix + stored
shape; full curator-shaped PUT persists reference/tags/charts/datasets,
admin allowed, non-owner 403, owner_email immutable, invalid payload 400)
-> nose2 73 OK. frontend +13 (model round-trip on the real fixture,
EditModeController create/unauthorized/anonymous/load/save/forbidden,
PermissionNotice link) -> yarn test 44 OK; yarn build OK.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Two staging blockers in /curator?edit=<id>:

1) "collections must be a `string` type, but the final value was:
   [\"MICCOM\"]" — curator state keeps collections/tags as ARRAYS and
   PaperInfoForm edits them as comma-separated strings, but its
   defaultValues joined only tags; collections passed the raw array into
   the yup string field. One missing join — also a latent CREATE-mode bug
   (re-editing an already-saved Paper Information section failed the same
   way); edit mode just surfaced it immediately. Both fields now join
   defensively; onSubmit still splits back to arrays, so the backend shape
   is unchanged and create mode behaves exactly as before.

2) "CSRF token missing or invalid" on Save Changes — the interceptor
   attached the token only when the module cache happened to be populated
   by AuthState's initial /me; any path where that fetch failed or the
   backend session store was replaced (e.g. staging rebuild between login
   and save) left the cache empty/stale with no recovery. The CSRF wiring
   is now self-healing, still same-origin-only, CSRF never disabled:
   - mutating same-origin requests fetch the token JUST IN TIME from
     /api/auth/me when nothing is cached (async request interceptor; /me
     is a GET, so no recursion);
   - a 403 "CSRF" response drops the cache so the user's retry refetches
     a fresh token for the new session.

tests: NEW CsrfIntegration.spec.js runs the REAL axios interceptor
pipeline against a stub adapter — Save Changes carries X-CSRF-Token from
the cached /me, from the just-in-time fetch when nothing cached it, and
re-fetches after a stale-token 403 (+3). NEW PaperInfoForm.spec.js loads
array-backed state (collections: ["MICCOM"]) — renders "MICCOM, PARADIM"
in the field and saves back clean arrays (+2). Backend proof that a
missing token is rejected and a valid one succeeds already exists
(test_update_paper.test_update_without_csrf_token_denied + owner tests).

Verified: yarn test 49 OK (was 44); yarn build OK; nose2 73 OK.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

1) Phantom empty "Extra Fields" row when editing tools/charts/datasets/
   scripts: stored records routinely carry a legacy placeholder row
   [{extrakey: "", extravalue: ""}] (not even this form's label/value
   keys), so the edit dialog seeded one blank row and the required-field
   schema then blocked saving. Shared handling in ExtraFieldInput:
   - cleanExtraFields() drops rows without a usable label AND value; used
     when seeding the field array from an existing item AND on submit in
     all four forms (both edit and add branches), so empty rows are never
     rendered unasked and never sent in payloads;
   - extraFieldsSchema (shared yup): an untouched empty row passes (it is
     filtered on submit), a half-filled row errors — replacing the four
     copies of label/value-required schemas, so a user-created blank row
     no longer blocks saving;
   - rows now take their default values from the field-array state (no
     defaults/index mismatch after filtering); the plus button still adds
     fresh rows; also fixes the value column showing the LABEL's error.
   Real saved extra fields load, edit and save exactly as before.

2) Overlapping First/Middle/Last labels with multiple PIs (and authors —
   same pattern in ReferenceInfoForm): the rows are plain nested Grids,
   and MUI v9's gap-based grid no longer gives nested non-container items
   the vertical padding the v4 negative-margin system did, so shrunk
   labels collided with the row above. Each map is now wrapped in a
   `container direction="column" spacing={2}` so every PI/author renders
   as a cleanly separated row with its remove button; data shape and
   submit behavior unchanged.

tests: NEW ExtraFieldInput.spec.js (+5: legacy placeholder row renders
nothing, empty/missing defaults render nothing, real fields still render
with values and survive mixed legacy junk, plus-button adds a row,
cleanExtraFields unit matrix); PaperInfoForm.spec.js +1 (two PIs render
as two distinct rows). yarn test 55 OK (was 49); yarn build OK;
nose2 73 OK (backend untouched).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The documented way to switch federation off was to set the variable to a
single space. It did the opposite. The value was `.strip()`ed and an empty
result was read as "the variable is not set", so the shipped list came back
and every peer on it stayed reachable. An operator disabling a feature got it
enabled.

Presence is now decided by `os.environ` membership, and only the CONTENT
decides what is allowed. Absent means registry plus shipped list; present
means exactly the origins named, and naming none -- "", " ", ",", or junk that
parses to nothing -- means an empty allowlist and no federation at all. An
empty allowlist can never widen into an open one: every ?server= is refused.

The Explorer had the mirror-image bug. It only adopted a published list if it
was non-empty, so a backend that had switched federation off still had its
shipped peers offered in the UI -- servers the backend would then refuse with
a 400. An empty published list is an answer and is now respected; the shipped
list is the fallback for exactly two cases, a failed request and an answer
that is not the documented shape.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The registry decides this server's outbound allowlist. It was fetched with
certificate verification, but the URL itself was only required to parse --
http:// was accepted. Verifying a certificate that is never presented buys
nothing: anyone on the path of a plaintext fetch could add themselves to the
list of servers this deployment will contact.

An http:// (or any non-HTTPS) registry URL is now not requested at all. The
result is "no registry", so the shipped list and an explicit
QRESP_FEDERATION_SERVERS both still apply, and a misconfiguration narrows
federation instead of weakening it. Redirects, the timeout and certificate
verification are unchanged, and the URL is still never logged -- it comes from
config.ini and may name an internal host.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…ast good answer

Stale-while-revalidate had no guard. Every reader of a stale entry called
spawn_background, so five readers arriving together on a record whose peer
caches had also expired started five refreshes and five rounds of peer reads
-- the exact multiplication the caching was added to prevent.

SingleFlight is the wrong tool here: it serialises callers who are all waiting
for the same answer, and nobody waits for a background refresh. RefreshGuard
instead lets exactly one reader start the work and tells the rest there is
nothing for them to do. Different keys never block each other, and the guard
is released in a finally, so an exception cannot strand one.

A failed refresh also used to overwrite the good stale answer with the
unavailable response it had just computed -- taking a true answer away from
readers because a peer was briefly unreachable. It now keeps serving the last
good result for the rest of its stale window, and the failure is recorded as a
45-second cooldown on that key instead, so one outage is not re-tried by every
page view and one new attempt is let through once it expires.

That distinction needed care: a refresh that preserves a good answer RETURNS
success and IS a failure, so `_refresh_and_report` reports the attempt's own
outcome rather than letting the caller infer it from the value served. Reading
it off the served value silently cleared the cooldown, and a test passed for
that wrong reason before this was split.

The guard holds an entry only while a refresh is in flight or a cooldown is
unexpired, and prunes on every call, so it tracks concurrent work rather than
every record ever viewed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

`after_gate` and `shown` were identical in every response ever produced, so
the pair could not report the one thing it existed for: how much the
three-result cap is discarding. `external_recommendations` had already
truncated the list, and the count was taken from the truncated result.

It now returns every candidate that cleared the gate when asked to, and the
caller applies the cap, so `after_gate >= shown` and the gap is visible. The
order is unchanged -- gate, sort, then cut -- and so is the default for other
callers.

The counts also vanished on a cache hit: they were computed live and never
stored, so the second view of a record answered without them and looked like a
different kind of answer. `reason` and `pipeline` are now stored alongside the
results, and a cached response explains itself exactly as the live one did.

The field is optional by design. An entry written before it existed has no
pipeline, so the key is omitted rather than invented, nothing crashes, and the
next real refresh fills it in -- the same migration-free pattern the
fingerprint and algorithm-version fields already use. Only booleans, a status
string and counts are stored: no title, no abstract, no provider body, no
credential.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

One class per defect, each reproducing it before the fix: the environment
allowlist across unset / valid / empty / whitespace / comma-only / junk, five
concurrent readers of a stale record refreshing once, a failed refresh
preserving the last good answer and cooling down for 45 seconds before exactly
one retry, `after_gate` exceeding `shown`, live and cached responses agreeing
on reason and pipeline, a legacy entry without a pipeline still serving, and
an http registry that is never requested.

The refresh guard shares the result cache's injectable clock, so a cooldown is
stepped over deliberately rather than waited out, and the peer caches are
cleared between views so the cooldown assertions measure the guard and not the
peer negative cache.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…tract

Descriptions left behind by earlier changes: "capped at five" and
"five-result cap" (the cap is three), authors listed as one of the independent
evidence families the gate reads (it is not evidence at all -- it only orders
candidates that already passed), and "set it to a single space" as the way to
switch federation off, which is now true rather than aspirational.

Adds what the four fixes changed: the exact allowlist table for absent /
named / empty / junk values, the pipeline field meanings with the
`after_gate >= shown` guarantee and the live-versus-cache parity, the
HTTPS-only registry row in the security order, and how a stale refresh is
guarded, preserved on failure and cooled down.

Documentation and comments only.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The AI action on an RCC candidate received the candidate's name, its
relative paths and the analyzer's structural sentences. Everything the
analysis had already read off the file server was discarded on the way:
_script_header() was defined and called from nowhere, README text reached
only Tool manifest parsing, notebooks were excluded from evidence reads
outright, and no function or class name was ever extracted. A Script whose
module docstring said exactly what it does was described from its filename.

project/evidence.py now extracts a structured, boundary-confined bundle per
candidate -- README, module docstring (ast), top-level def/class NAMES,
notebook MARKDOWN cells, manifest lines -- and curation.py attaches it as
`ai_sources`. The paper's title and abstract travel as `paper_context`, and
the prompt states an evidence hierarchy that forbids claiming what a script
computes or a chart shows on the strength of the abstract alone.

What is structurally impossible, not merely discouraged: raw dataset values,
image bytes, notebook code cells/outputs/attachments, function bodies and
string literals. No extractor for any of them exists. A Python file that
does not parse yields nothing rather than being regex-guessed.

Evidence never crosses a boundary, so a sibling dataset's README cannot
describe this one. Credential-shaped values are redacted before the bundle
is built and again on the way out, because it round-trips through the
browser. Budgets are explicit: 1200 chars per source, 3000 per candidate,
8 sources, and a read plan spent ROUND ROBIN across candidates so one large
folder cannot leave every later candidate's README unfetched.

_script_header is now on the real Script path and delegates to the shared
extractors, so the Details panel and the AI bundle cannot disagree about
what a file's header is.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

aiItem() built `context` from draft.readme + draft.description -- the
curator's own answer to the very field the model was being asked to fill.
A filled field produced a paraphrase of itself; an empty one produced
nothing but the analyzer's structural sentences. It also made any benchmark
against curator text self-fulfilling.

The request now carries the candidate's structured `ai_sources` and
`inventory`, plus the paper's title and abstract as background. The server
dropped `context` from its allowlist in the previous commit, so an older
client cannot reinstate the leak.

The consent dialog was also no longer true: it promised no notebook
contents while the payload now carries notebook markdown, and said nothing
about the paper background. It now itemises the ACTUAL source list for that
candidate -- type and path, one line each -- and warns, before the request
is spent, when a candidate has no readable text and the answer will be
"not enough evidence".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

65 new backend tests and 5 new frontend ones, over the properties that
matter rather than the shape of the payload:

- a sibling's README/docstring never describes this candidate, and
  `scripts/analysis2` is not inside `scripts/analysis`;
- _script_header is actually CALLED (a spy, because "defined and unused"
  is exactly the bug being fixed);
- top-level names only -- no bodies, no literals, no nested methods, no
  private helpers, and nothing at all from a file with a syntax error;
- notebook markdown only -- no code, no outputs, no base64 attachments --
  and a corrupt notebook skips its own evidence, not the analysis;
- a Chart with only an image gets no sources, so abstention is the
  correct answer;
- title/abstract travel but are not artifact evidence, asserted against
  the prompt's own wording;
- Tool keywords are dropped server-side and a Tool is never even asked;
- secret redaction, per-source/per-candidate/per-request caps, forged
  source types and paths, prompt injection, confidence clamping;
- exactly one candidate per provider call, one quota unit, and none spent
  on a rejected request.

Existing tests that pinned the OLD contract are updated, not deleted:
`payload["item"]` became the `paper_context`/`artifact`/`sources` bundle,
and the root requirements.txt is no longer fetched because a root file
belongs to no boundary and nothing ever consumed it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Every sampled candidate is now asked TWICE -- once with the pre-change
input (name, paths, structural sentences) and once with the shipped bundle
-- so the difference is paired on the same candidate instead of on two
different samples. The structural sentences travel as
`artifact.structure_notes`, not as a source, so a description copied out
of them does not score as grounded.

Reported per record type per mode: description groundedness (against the
EVIDENCE, with the paper's abstract deliberately excluded from the
denominator), a usefulness floor, keyword concept precision/recall,
generic-term ratio measured before the server's stopword filter, and
abstention correctness -- where `missed_abstention` (described an artifact
with nothing to describe from) is the failure this change targets.

Leakage work:

- the reference corpus is the published corpus MINUS Qresp's own QA/test
  records, each exclusion printed with the rule that fired it. The first
  rule was over-eager and matched "Testing the limits of DFT for water";
  a QA word now has to be used as a label, not as a sentence;
- the target record's curated description AND keywords are scrubbed from
  every source excerpt, which makes keyword recall an inference test
  rather than a copying test;
- the leave-one-record-out vocabulary is unchanged and still holds out
  the target;
- adding `paper_context` opened a real channel: a paper titled "Band
  structure of monolayer transition metal dichalcogenides" contains two
  of its own artifacts' reference keywords. Deleting the title would
  benchmark a product that does not exist, so those keywords are counted
  and recall is reported a second time without them;
- a unit whose FINAL payload still contains the curated description is
  dropped and never called, with the reason printed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…mark

RCC_FOLDER_ANALYSIS.md now shows the actual request shape, a per-kind
table of which source types each record type may carry, the budgets, and
the two facts a reader most needs: nothing the curator typed is sent, and
abstention is a correct answer for a Chart that is only an image.

AI_ASSIST_EVALUATION.md documents the two evidence modes, what each metric
means, and the paper-title caveat -- including why it is reported rather
than scrubbed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The prompt asked the model to return an empty description when `sources` is
empty. That is a request, not a guarantee. The server called Gemini anyway,
spent a quota unit anyway, and returned whatever came back -- which for a
Chart holding nothing but an image meant a caption assembled from the file
name and the paper's abstract, the one thing the prompt most explicitly
forbids.

describe_candidates() now decides abstention itself, from the evidence,
after authentication/CSRF/consent/one-candidate (all unchanged) and BEFORE
the provider configuration is read:

    {"suggestions": {}, "no_suggestion": ["chart-0"]}   HTTP 200

No new API field: `no_suggestion` already carries this, and the browser
already handles it. The answer is the same on a server with no API key,
because whether a candidate can be described is a property of the folder,
not of the provider -- a candidate that DOES have evidence still gets 503
there.

_sanitize_sources() also had no idea what kind of candidate it was
validating, so the global type allowlist let a tampered client hang a
`docstring` on a Chart. It now takes the kind and filters against
evidence.accepted_source_types(), which is the single table both directions
read: the extractors decide what to produce from it, the endpoint decides
what to accept from it, and AI_SOURCE_TYPES is derived from it rather than
repeated beside it. A bundle filtered to nothing takes the abstention path.

swagger.yml's enum cannot express this -- it knows the seven type names, not
which kind may hold which -- so it stays a first gate only.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

`no_suggestion` now carries two different situations and the curator needs
to tell them apart: the server declined to ask because the candidate has no
evidence of its own, or it asked and the provider had nothing usable. One
is fixed by adding a README to the folder; the other by trying again.

The message is chosen from the candidate's own `ai_sources`, which the
browser already holds -- no new API field, so the two cannot drift apart.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Written before the fix; 16 of them failed against 6893f7f.

Backend (28 new). Every abstention case watches the provider AND the quota
counter, because asserting only "no provider call" would still pass if the
curator had been charged for a request that was then not made:

- empty sources, for all four kinds, and sources that sanitize away to
  nothing -- 200, empty suggestions, id in no_suggestion, zero calls;
- a Chart carrying only a forged docstring, a Dataset only python_symbols,
  a Script only notebook_markdown, a Tool only python_symbols -- all
  filtered to nothing, all abstaining;
- a mixed bundle keeps exactly the types its kind can carry;
- a Chart with a real README and a Script with docstring + symbols still
  make exactly one call and spend exactly one quota unit;
- an unconfigured provider abstains on no evidence and still 503s on real
  evidence;
- auth, CSRF, consent and the one-candidate rule are still enforced on the
  no-evidence path, which is not a shortcut around them;
- the abstain path writes nothing and logs no evidence text;
- the analyzer never emits a source its own kind's filter would reject.

Also pins that swagger.yml parses and that its enum matches the code. An
unquoted JSON brace in a description opens a YAML flow mapping and breaks
the spec, which surfaces as every test module failing to IMPORT -- a
confusing signal for a typo in one string.

Frontend (3 new): the evidence-based notice, the provider-had-nothing
notice, and that an abstention leaves the curator's typed value alone and
adds nothing.

Two existing Tool tests supplied a Script fixture's docstring as their
evidence; they now supply a manifest and a declaration, because a Tool
cannot carry a docstring and would otherwise (correctly) abstain.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Records the exact response, that it costs no quota and makes no provider
call, that it answers the same way on a server with no API key, and that
the per-kind source table is enforced on the way in as well as on the way
out.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

A Chart whose caption the analyser could not determine kept its analysis-time
"Needs input" chip AFTER the AI filled the field -- while the card header,
which reads the draft, correctly counted the field as no longer missing. The
same staleness would have let a "High evidence" chip, earned by a file path
the analyser detected, vouch for a path a curator had typed over it.

The cause was one line reading the wrong thing: the chip rendered
`candidate.field_evidence[field]` whenever the field was non-empty.
`field_evidence` is a statement about `candidate.proposal`, frozen at analysis
time; nothing re-read it, so it described a value that was no longer there.

Three facts were being conflated -- what the analysis proposed, what the field
holds now, and how strong the analysis' evidence was. `valueState`,
`evidenceChipFor` and `suggestionApplied` in Utils/artifactFields.js are now
the single place they are combined:

  blank      -> no chip. The asterisk, the helper text and the header's
                missing count are the three required indicators; a fourth is
                noise, and flagging OPTIONAL fields this way was a bug once
                already.
  unchanged  -> the analysis' own high/medium standing.
  changed    -> no chip. Nothing verified this value.

`needs_input` is therefore unreachable in either direction rather than
special-cased.

The AI panel's "not applied" was a hardcoded literal that never changed. It
and both Use buttons are now DERIVED from the draft, so applying a suggestion
says "applied", editing or clearing the value says "not applied" again, and
the button stops telling the curator their text is being protected from the
AI when the text in the field is the AI's own.

Nothing about what is sent, saved, added or published changes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The Explorer is about to open on results instead of on a node picker, so
"which server" stops being something a visitor types and becomes something
the deployment has to answer. It is answered here, beside the allowlist,
rather than in React -- a default hardcoded in a page would be a second copy
of the federation config, and the first copy already drifted once.

`/api/federation/servers` gains `default_server`. Additive: a client that
only reads `servers` is unaffected.

QRESP_DEFAULT_EXPLORER_SERVER names it. The value goes through the same
`parse_origin` as every other origin -- so a trailing slash, a mixed-case
host and an explicit :443 all resolve to the spelling the allowlist holds --
and is then checked for MEMBERSHIP. Naming a server here can pick among the
federated ones and can never add one; an origin outside the allowlist is
ignored with a log line rather than obeyed, because a default the allowlist
refuses would send every first-time visitor into a 400 naming a server they
never chose.

Without the variable: the first origin in the published (sorted) order --
deterministic, and visibly the first row.

An empty allowlist yields "", which is an answer ("this deployment federates
with nobody") and not a failure. No SSRF, HTTPS, literal-address or DNS check
is touched: this only chooses among origins those checks already permit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Clicking EXPLORER asked a question before showing anything: pick a Qresp
node, then search. Answering it wrong -- Duke, currently unreachable --
produced a blocking "Search Error!" modal on top of a page reading "0 Records
Available", which is exactly what a healthy but empty node also says.

`/explorer` now redirects server-side to the default server's results, so the
navbar link, a typed URL, a refresh and the back button all behave the same.
The target comes from `/api/federation/servers`; no host is named in this
file, and a `default_server` the published list does not contain is refused
here too rather than taken on trust.

Federation is not reduced to one node: `/explorer?choose=1` still offers the
picker, `/search?servers=a,b` is unchanged, and each record keeps its own
source server in its detail link.

The search page had one failure mode for two situations. Now:

  - some nodes failed  -> results from the ones that answered, plus a
                          non-blocking warning naming the ones that did not;
  - every node failed  -> an in-page unavailable panel with Retry that says
                          this is a connection problem, NOT an empty node;
  - navigating         -> an explicit "Searching…" state, because Next keeps
                          the previous page mounted while it fetches and the
                          stale count would read as the new one.

No blocking modal is raised for a search failure at all any more.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

All of these fail against 7379e69 and pass after the two fixes above.

Folder Analysis (7 integration + 15 unit). The integration tests use the real
analyzer's own field_evidence for a chart folder -- imageFile high, everything
a human supplies needs_input -- which is the shape that produced the report:

  - the stale "Needs input" chip is gone once AI fills caption and keywords,
    and the header goes 3 -> 1 on the same click;
  - the panel says "applied" only when the suggestion's values are actually
    in the fields, and goes back to "not applied" when one is edited;
  - clearing an applied value restores the missing count and re-enables Use;
  - High evidence disappears when the value stops being the analysed one;
  - applying adds, saves and publishes nothing (2 posts: analyse, describe);
  - closing and re-analysing leaves no applied state on the same candidate id.

Federation (10). The default is published, is always one of `servers`, is
normalized before it is compared, is REFUSED when it names an origin outside
the allowlist (falling back to the first listed one), never widens the
allowlist, and is "" when this deployment federates with nobody.

Explorer (9) and search (8). The redirect target comes from the backend and
only the backend; no peer is contacted while deciding it; an unlisted default
is refused; an empty or unreachable federation shows an unavailable page
instead of redirecting; `?choose=1` still gets the picker without spending a
request. The search page keeps partial results with a non-blocking warning,
shows an unavailable panel with Retry only when everything failed, never
raises a modal, never shows "0 Records Available" while loading, and keeps
each record's source server in its link.

One test reads pages/explorer.js and asserts no server hostname and no record
count is hardcoded in it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

What the Explorer's default server is, the table of what each value of the
new variable does, why an origin outside the allowlist is ignored rather than
obeyed, and that the picker and multi-server URLs are still reachable.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The panel had two states for a suggestion that can offer two things. Using
only the keywords left the button reading "Applied to Keywords" directly
below a header still reading "not applied" -- the same kind of self-
contradiction the stale evidence chip produced.

`suggestionState(kind, draft, offers)` in Utils/artifactFields.js is a small
pure helper over what the suggestion ACTUALLY offered: an entry with no target
field (a Tool has no keyword field) or an empty value was never an offer and
cannot hold the state back. So a description-only suggestion is `applied`
after one click rather than stranded waiting for a second field that does not
exist.

  none of what it offered is in place  -> not applied
  some                                 -> partially applied
  all                                  -> applied

Still derived from the draft on every render, never stored, so editing or
clearing an applied value walks the state back on its own. The state is
spelled out in text; the colour is only a second cue.

The per-field "Applied to ..." buttons, the stale-chip fix and the
evidence-only-while-unchanged rule are untouched, and nothing here saves,
publishes or adds anything.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

A Qresp node is asked for four endpoints and they are not equal:
/api/search fills the results table, while /api/collections, /api/authors and
/api/publications only fill dropdowns in AdvancedSearch. The loop treated
them as one list, `break`ing on the first failure of either kind and marking
the whole SERVER failed.

So a node whose records had already loaded, but whose authors list 404'd, was
announced as one whose "records are missing from these results" -- directly
above the rows it had served. Worse, `total` was `failed.length >=
servers.length`, so with a single configured node that one broken auxiliary
endpoint replaced the whole page with an unavailable panel while its records
sat in `data`.

The core endpoint is now fetched first and staged in a local, committed only
once it has actually arrived; its failure drops that node's records and skips
its filters entirely. The auxiliary endpoints are then fetched independently
-- no `break`, because one being down says nothing about the other two, and
the old flow discarded filters that had nothing wrong with them.

`error.failed` (records missing) and `error.filters` (records fine, filters
short) are separate, and total failure is measured on how many nodes actually
produced records rather than on a count of nodes with something wrong.
`error.is`/`error.msg` are kept for older readers.

The filter notice names the endpoints: "Records were loaded, but some search
filters are unavailable from: https://x (authors, collections)."

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Searching PaperStack and Duke together put an un-dismissable dialog over
PaperStack's perfectly good matches. `/search`'s SSR load had already learned
to tell a failed node from a failed filter and to say so beside the results;
`AdvancedSearch.onSubmit` had not, and still called the global `setAlert()`
on any server error.

The cause was ownership. The component ran the search AND decided what the
page said about it, so its only vocabulary for "one of two nodes is down" was
a modal. The results live in `pages/search.js`, so the status that describes
them now lives there too: the component reports `{papers, failedServers,
totalFailure, retry}` and the page decides.

  every node answered   -> results replaced, no notice
  some answered         -> only those committed, inline warning naming the
                           others; the good matches are never discarded
  none answered, results
    already on screen    -> nothing is committed, inline error saying the
                           previous results are still shown
  none answered, nothing
    on screen            -> inline error, and the record count is withheld
                           rather than reading "0 Records Available"
  answered with 0 rows   -> an ordinary 0 Records Available, not a failure

Results are staged per server and committed only after every node has been
asked, so a late failure cannot land after a partial commit, and a total
failure never calls setData({}) over results that are still valid.

Retry re-runs the criteria and server list captured when the search started,
not whatever is in the form by the time it is pressed. `showLoader`/
`hideLoader` are paired in a `finally`, a submit already in flight cannot be
started again, and a result arriving after unmount is dropped. The thrown
error goes to the console; the page is told only WHICH server failed, so no
host or stack reaches the screen.

The SSR notices (`error.failed`, `error.filters`) and this runtime one are
separate state and can be shown together -- they describe different events.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

On the evaluated 65-record server, recommendations were being justified to
readers with `university wisconsin-madison`, `argonne national lab`,
`microsoft powerpoint`, `fig4`, `fig5`, `table1`, `represented`,
`positioned`, `individual`, `principal`, `conventional` and `highlighting`.
Four separate mechanisms produced them, and none was a missing blocklist
entry:

  * `LONG_TECHNICAL_LENGTH = 9` made any plain nine-letter word subject
    vocabulary. The number could not be tuned, because the string carries no
    signal: `chalcogenide` and `conventional` are the same shape and, on 65
    records, the same rarity.
  * "a digit or a hyphen makes it technical" admitted `fig4` and `panel-a`
    alongside `g0w0` and `bethe-salpeter`.
  * `facilityName` was read as a METHOD, and a shared method plus any topic
    word was the strongest verdict the gate has -- so a shared employer was
    a strong recommendation.
  * `packageName` took the same path, so sharing a slide deck was strong
    evidence.

The fix is PROVENANCE. `Profile.term_sources` records whether a word arrived
from a title, a curated tag/property/keyword, an abstract, prose, software, a
technique or an organisation, and the gate asks. What separates
`chalcogenide` from `conventional` is not the word, it is that one of them is
in somebody's title.

  * `has_technical_shape` replaces the length rule: a plain word now proves
    nothing by spelling. It qualifies by being stated deliberately instead.
  * `pair_specific_terms` decides specificity per PAIR, because provenance is
    asymmetric -- a term can be in one title and the other's abstract, which
    is the "concept confirmed between a title and an abstract" case.
  * STRONG by terms additionally requires a shared term BOTH sides state
    deliberately. Prose agreement is a medium.
  * `is_structural` rules out document furniture by name, so the shape rule
    stays available to real formulas.
  * `is_organizational` keeps facilities out of terms, methods, similarity
    and reasons entirely -- they are read only so they are visibly not
    scored.
  * software and technique overlap is capped at MEDIUM for ever and generic
    tooling is not even named, so it can corroborate a subject but never
    establish one.
  * a shared collection stopped being evidence; it was pairing with any one
    weak signal to open the gate.
  * the author tie-break is gone from `rank`: the gate was topic-only but the
    ORDER was not, so a common PI could still decide which three a reader saw.

`is_ordinary` now also tests the stem, so participles nobody listed
(`highlighting`, `represented`, `positioned`) are caught by entries already
there rather than by growing the list one inflection at a time.

External candidates go through the identical gate; the provider recommending
something has never been evidence and still is not.

ALGORITHM_VERSION 2 -> 3 and FINGERPRINT_VERSION 1 -> 2 so no cached verdict
computed by the old gate is reused.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

… return

30 new tests, plus two existing ones updated to the contract they now
contradict:

  * `test_the_same_tool_on_the_same_topic_is_strong` asserted the door
    `facilityName` walked through. It is now
    `test_the_same_tool_is_never_more_than_medium`, and still asserts the
    pair passes -- on its shared curated topic, which should have been
    carrying it all along.
  * the specificity test asserted `is_specific("gadgetite")`, i.e. the
    plain-word length bar. It now asserts the opposite, alongside
    `conventional`, and keeps the shape cases (`rareword resonance`,
    `bivo4`).

The new file covers: long ordinary words have no technical shape; their
inflections are ordinary via the stem; the same long word counts from a title
and does not from prose; document furniture is recognised and real formulas
are not mistaken for it; facilities never become method terms and a shared
employer cannot pass a pair; generic software proves nothing and no method
evidence is ever strong; STRONG needs a deliberate source on both sides while
a tag on one side and a title on the other still counts; collections are not
evidence; removing or replacing every author changes neither gate nor order,
and author overlap is absent from the sort key; caps hold at three and zero
is an acceptable answer; external candidates face the identical gate; and the
cache version moved with the algorithm.

One test builds a corpus of four unrelated subjects that share an employer, a
slide deck, a programme and a boilerplate paragraph, asserts none of the
named polluted strings can appear in any reason, and asserts the right answer
is zero recommendations.

Fixtures are synthetic: the module hardcodes no DOI, title, material or
facility, so the corpus is invented and the real thresholds still run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Behaviour-neutral. Every gate verdict over the real 65-record corpus (3,200
pairs), every shown list, score and reason is byte-identical before and after
-- captured and diffed, and asserted from outside by the new neutrality
tests.

The quality rework stopped the gate reading authors and collections, but left
the machinery that computed them and the comments that described it. Traced
each symbol to its consumers; these had none:

  * `shared_fields`, `topic_terms`, `topic_overlap` -- computed in `assess`
    and never read. `topic_overlap` was the corroboration test for the strong
    method verdict that no longer exists.
  * `author_overlap` on `Assessment`, and `shared_authors` that fed it -- set,
    never read, by anything in the repository.
  * `Profile.author_keys` / `author_names` and `author_key()` -- existed only
    to build that count.
  * `Profile.field_terms` and `add_field()` -- fed only `shared_fields`, so
    `collections` is no longer read at all.

`Profile.authors` STAYS: `related._result` renders it beside each
recommendation, which is the one thing authors are for.

The fingerprint hashed authors and collections too, so correcting the
spelling of a name or filing a record under a second programme threw away a
cached Semantic Scholar answer and paid for a fresh request to rebuild an
answer that could not have changed. Both are dropped;
FINGERPRINT_VERSION 2 -> 3 so entries hashed under the old allowlist are a
clean miss rather than a silent never-match. `facilityName` stays hashed --
it never becomes a term, but it decides which terms are excluded as
organisational, so editing one CAN change an answer.

ALGORITHM_VERSION is deliberately NOT bumped: no verdict changes, so cached
results stay valid.

Documentation now says what the code does. Removed: "a shared author ... only
orders candidates that have already passed" from the module docstring, the
tie-break claim in `rank`, the `author_overlap` note on `Assessment`, and a
comment pointing at a `supporting_note` that never existed. The federation
field-allowlist note now states where its correspondence with the profile and
the fingerprint is exact and where it deliberately is not.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

19 new tests stating what a recommendation may not depend on, all from the
outside -- candidates, order, reasons, verdicts and the fingerprint -- so the
implementation stays free to drop the state entirely:

  * removing or replacing every author changes nothing a reader sees, a
    shared author cannot break a tie, and no reason ever names a person;
  * removing or replacing every collection changes nothing, a shared
    collection cannot pass a pair, and no reason names one;
  * a candidate's authors are still returned by `_result` for display, and an
    external candidate's authors and fields change neither verdict, score nor
    reasons;
  * author and collection edits do NOT move the fingerprint, while every
    input that can change an answer -- including a facility name -- still
    does;
  * FINGERPRINT_VERSION moved and ALGORITHM_VERSION did not;
  * the quality guarantees are re-asserted so this cleanup cannot quietly
    cost them: organisations, office software and figure/table tokens stay
    out, a shared tool alone still fails, the cap holds at three, zero is
    still an answer, and external candidates face the same gate.

Three existing tests pinned the behaviour that was just removed and were
asserting things that are no longer true:

  * `test_author_key_survives_initials_and_middle_names` tested a helper that
    existed only to compare two people. Author matching is gone, so it now
    asserts its absence and points at the contract that replaced it.
  * `test_every_field_a_recommendation_depends_on_changes_it` listed authors
    and collections among the fields a recommendation depends on. They moved
    to a new "does not invalidate a cache" case beside it.
  * `test_every_scoring_field_forces_a_refetch` asserted end to end that
    editing collections re-queries the provider -- the wasteful behaviour
    itself. Collections and authors moved to
    `test_metadata_that_scores_nothing_does_not_refetch`, which asserts no
    provider call and an unchanged fingerprint.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Behaviour-neutral. Every gate verdict over the real 65-record corpus (3,200
pairs), every shown list, score and reason is byte-identical before and after.

Documentation first. The module still described authors as something that
"orders candidates that already passed", which stopped being true when the
tie-break was removed. The history is worth keeping, so it is now stated as
history: authors are display metadata, they take no part in the gate, the
score, the evidence, the order or the tie-break, and the note in `assess`
explains what they used to do.

`build_internal_profile` claimed to read `collections` "as broad FIELDS" and
"author names". Neither is true: `collections` has not been read since the
state that consumed it was deleted, and authors are copied onto the Profile
without being scored so that `related._result` can render them. The docstring
now lists the input fields exhaustively and says which of them is read only
in order to be excluded (`facilityName`).

`FAMILY_AUTHORS` existed, by its own comment, "so that a test can assert its
absence". That is backwards -- it left a family constant no evidence uses
sitting there for a future change to reach for, and the test passed by
checking an unused string was missing. The test now asserts the outcome: no
`"authors"` family in the evidence, AND the same verdict, score and reasons
whether the two records share an author or not.

`MODERATE_TEXT_SIMILARITY` was NOT unused, which is worth recording: `rg`
found one consumer in `tools/eval_core.py`, printing a "moderate bar" in the
sentence that explains why a pair was rejected. But that bar stopped gating
anything when `topic_overlap` was deleted, and the same sentence still said
"no shared author" -- two claims about a gate that no longer works that way.
Correcting the sentence to cite the bar that does apply is what left the
constant with no consumers, and only then is it removed.

No verdict, threshold, provenance rule, cap, fingerprint or version changes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>